Systems, devices, and methods for image analysis

ABSTRACT

Embodiments of a method for performing image analysis include accessing a digital image of a subject, the digital image comprising a portion of a target substrate of the subject, the digital image comprising a portion of a calibration device; detecting a position of the calibration device by searching the digital image for a predetermined background color; detecting a plurality of color chips on the calibration device; mapping a position of the plurality of color chips on the digital image, each of the plurality of color chips comprising a predetermined chip color; comparing at least a portion of the plurality of color chips with a predetermined color standard; and altering the digital image, based on the comparison, to provide consistent image parameters with other images of the subject.

RELATED PATENT APPLICATIONS

The present application claims priority from U.S. Provisional PatentApplication No. 61/531,280 filed on Sep. 6, 2011 and from U.S.Provisional Patent Application No. 61/545,920 filed on Oct. 11, 2011,both of which are incorporated herein by reference in their entireties.

FIELD OF THE INVENTION

The present invention relates in general to systems, devices, andmethods for performing image analysis.

BACKGROUND INFORMATION

Countless individuals all over the world seek to improve their physicalappearance and health through the use of medical and cosmetic products,such as color cosmetics, skin care products, body care products, haircare products, etc. Many of these products are available through retailstores and/or pharmacies, where a live consultant may be available toassist with the selection of the most appropriate product. In someinstances, virtual consultations may be provided in the retail store toassist with the product selection. However, oftentimes, theseconsultations may be difficult and/or embarrassing for the user tocomplete at the retail establishment. Further, oftentimes the user hasno place to store, share, compare, or otherwise utilize the consultationanalysis and/or product recommendations.

In addition, there is a continuing desire to provide systems, devices,and methods that enable a consumer to track his/her usage of cosmeticproducts at dates, times, and locations of his/her choice. Stillfurther, there is continuing desire to provide systems, devices, andmethods that provide ubiquitous access to product recommendations, dataassociated with a consumer's use of cosmetic products, and/or dataassociated with the state of the consumer's skin. Still yet further,there is a continuing desire to provide systems, devices, and methodsthat can accommodate the use of different image capture devices atdisparate locations as part of the tracking and/or consultationexperience.

SUMMARY OF THE INVENTION

Embodiments of a method for performing image analysis include accessinga digital image of a subject, the digital image comprising a portion ofa target substrate of the subject, the digital image comprising aportion of a calibration device; detecting a position of the calibrationdevice by searching the digital image for a predetermined backgroundcolor; detecting a plurality of color chips on the calibration device;mapping a position of the plurality of color chips on the digital image,each of the plurality of color chips comprising a predetermined chipcolor; comparing at least a portion of the plurality of color chips witha predetermined color standard; and altering the digital image, based onthe comparison, to provide consistent image parameters with other imagesof the subject.

Embodiments of a system for performing image analysis include a memorycomponent that stores logic that, when executed by the system, causesthe system to perform at least the following: access a digital image ofa subject, the digital image comprising a portion of a target substrateof the subject, the digital image comprising a portion of a calibrationdevice; detect a position of the calibration device by searching thedigital image for a predetermined background color; detect a pluralityof color chips on the calibration device; map a position of theplurality of color chips on the digital image, each of the plurality ofcolor chips comprising a predetermined chip color; compare at least aportion of the plurality of color chips with a predetermined colorstandard; and alter the digital image, based on the comparison, toprovide consistent image parameters with other images of the subject.

The above and other aspects and features of the present invention willbe apparent from the drawings and detailed description which follow.

BRIEF DESCRIPTION OF THE DRAWINGS

The following description of various embodiments of the presentdisclosure can best be understood when read in conjunction with thefollowing drawings:

FIG. 1 depicts a system illustrating various components that may beutilized for providing virtual consultation, according to someembodiments disclosed herein;

FIG. 2A depicts a computing device that may be utilized for providing avirtual consultation, according to some embodiments disclosed herein;

FIG. 2B depicts a calibration device that may be utilized for providinga color calibration of images for a consultation, according to someembodiments disclosed herein;

FIG. 2C depicts a product package, which may include a calibrationdevice and a cosmetic product, according to some embodiments disclosedherein;

FIG. 3 depicts a user interface for providing a product and/or treatmentconsultation, according to some embodiments disclosed herein;

FIG. 4 depicts a user interface for creating a profile for performing aproduct and/or treatment consultation, according to some embodimentsdisclosed herein;

FIG. 5 depicts a user interface for selecting a consultant, according tosome embodiments disclosed herein;

FIG. 6 depicts a user interface for viewing a profile of a selectedconsultant, according to some embodiments disclosed herein;

FIG. 7 depicts a user interface for scheduling a consultation, accordingto some embodiments disclosed herein;

FIG. 8 depicts a user interface for beginning a consultation on anuploaded image, according to some embodiments disclosed herein;

FIG. 9 depicts a user interface for identifying wrinkles on the targetsubstrate, according to some embodiments disclosed herein;

FIG. 10 depicts a user interface for identifying spots on the targetsubstrate, according to some embodiments disclosed herein;

FIG. 11 depicts a user interface for providing a communication portalwith a consultant, according to some embodiments disclosed herein;

FIG. 12 depicts a user interface for providing treatment and/or productrecommendations to a user, according to some embodiments disclosedherein;

FIGS. 13A-13D depict a plurality of images that illustrates colorcorrection, according to some embodiments disclosed herein;

FIG. 14 depicts a plurality of images that illustrates final resultsfrom color correction;

FIGS. 15A and 15B depict a plurality of images that illustrates featuredetection and spatial image adjustment, according to some embodimentsdisclosed herein;

FIG. 16 depicts a flowchart for providing a virtual consultation,according to some embodiments disclosed herein;

FIG. 17 depicts another flowchart for providing a virtual consultation,according to some embodiments disclosed herein;

FIG. 18 depicts a flowchart for providing a recommendation to a user,according to some embodiments disclosed herein;

FIG. 19 depicts another flowchart for providing a recommendation to auser, according to some embodiments disclosed herein;

FIG. 20 depicts a flowchart for capturing an image of a user and astandard calibration device, according to some embodiments disclosedherein;

FIG. 21 depicts a flowchart for altering an image for analysis,according to some embodiments disclosed herein;

FIG. 22 depicts a flowchart for performing color correction, accordingto some embodiments disclosed herein;

FIG. 23 depicts a flowchart for detecting a calibration device,according to some embodiments disclosed herein;

FIG. 24 depicts a flowchart for detecting a resolution indicator,according to some embodiments disclosed herein;

FIG. 25 depicts a flowchart for separating a resolution indicator,according to some embodiments disclosed herein;

FIG. 26 depicts a flowchart for performing image resolution estimation,according to some embodiments disclosed herein;

FIG. 27 depicts a flowchart for isolating the calibration deviceuniquely, according to some embodiments disclosed herein;

FIG. 28 depicts a flowchart for performing phase 1 image colorcorrection, according to some embodiments disclosed herein;

FIG. 29 depicts a flowchart for performing phase 2 image colorcorrection, according to some embodiments disclosed herein;

FIG. 30 depicts a flowchart for performing phase 3 image colorcorrection, according to some embodiments disclosed herein; and

FIG. 31 depicts a flowchart for performing phase 4 image colorcorrection, according to some embodiments disclosed herein.

DETAILED DESCRIPTION

The present invention will now be described with occasional reference tothe specific embodiments of the invention. This invention may, however,be embodied in different forms and should not be construed as beinglimited to the embodiments set forth herein. Rather, these embodimentsare provided so that this disclosure will be thorough and complete, andto fully convey the scope of the invention to those skilled in the art.

The skilled artisan will readily appreciate that the devices and methodsherein are merely exemplary and that variations can be made withoutdeparting from the spirit and scope of the invention. It is also to beunderstood that the terminology used herein is for the purpose ofdescribing particular embodiments only and is not intended to belimiting. As used in the description of the invention and appendedclaims, the singular “a,” “an,” and “the” are intended to include theplural forms as well, unless the context clearly indicates otherwise.

Unless otherwise defined, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of skill in the artto which this invention pertains. The terminology used in thedescription of the invention herein is for describing the particularembodiments only and is not intended to be limiting of the invention.

The term “cosmetic products” means any good that may be used to improveand/or alter the appearance and/or health of a user. Cosmetic productsinclude, but are not limited to, products for treating hair (human, dog,and/or cat), including, bleaching, coloring, dyeing, conditioning,growing, removing, retarding growth, shampooing, styling; deodorants,and antiperspirants; personal cleansing, including the washing,cleaning, cleansing, and/or exfoliating of the skin, including the face,hands, and body, optionally in concert with a cleaning implement,including a sponge, woven substrate, or non-woven substrate; colorcosmetics; products, and/or methods relating to treating skin (human,dog, and/or cat), including application of creams, lotions, and othertopically applied products for consumer use; and products and/or methodsrelating to orally administered materials for enhancing the appearanceof hair, skin, and/or nails (human, dog, and/or cat); and shaving,including razors and other shaving devices as well as compositionsapplied before or after shaving.

The term “kiosk” means any stand alone device, electronic of otherwisethat is specifically and exclusively configured for providing an audioand/or visual consultation to a user. The consultation may additionallyinclude providing a calibration device and/or other a previouslyinaccessible item (such as a cosmetic product and/or calibration device)to a user. In such instances, the item may be purchased and/or dispensedfrom the kiosk. The consultation may additionally include providing anoption for a virtual selection and/or purchase of one or more cosmeticproducts. In such instances, the cosmetic products may be shipped to theuser at a specified location or the kiosk may provide directions to abrick and mortar retail location where the cosmetic product may bepurchased or is awaiting pickup. Kiosks may be provided in a widevariety of shapes and sizes and may be located in retail locations, suchas shopping malls, medical offices, etc.

The term “consultation data” means any information that may be providedto a user as part of a consultation.

The term “general purpose computer” means any computing device that canreceive and store different applications and/or logic for execution.

The term “social network” means any system for providing an electronicforum for users to interact with other users. Some non-limiting examplesof social networking systems suitable for use with the present inventionare described in USPNs 2011/0093460 and 2010/0132049.

The term “cosmetic products” means any product that may be applied to atarget substrate to alter the appearance and/or health of the targetsubstrate.

The following description includes reference to different colors andcolor spaces. In that regard, the following conventions may be followed.These terms may be defined with additional language in the remainingportions of the specification.

The term “LAB color space” refers to a color measurement conventionwherein the L value, A value and B value may be plotted in a threedimensional space using polar components where dimension L defineslightness and A and B for the color-opponent dimensions, based onnonlinearly compressed CIE XYZ color space coordinates.

The term “RGB color space” refers to any additive color space based onthe red-green-blue (RGB) color model. A particular RGB color space isdefined by the three chromaticities of the red, green, and blue additiveprimaries, and can produce any chromaticity that is the triangle definedby those primary colors. The complete specification of an RGB colorspace also requires a white point chromaticity and a gamma correctioncurve.

“Target substrate” is a portion of a user's body, including, withoutlimitation, skin, hair, lips, nails, eyes, and teeth, to which portionsample areas may be color corrected based on a comparison of thecalibration device to known color and/or resolution standards. In someembodiments, the target substrate is the face and in some embodimentsthe target substrate is one side of the face (e.g., substantially eitherthe right or left side of the face including the eye and cheek thereof).

Example System

Referring now to the drawings, FIG. 1 depicts a system 10 illustratingvarious components that may be utilized for providing a beautyconsultation to a consumer from one or more computing devices. While thecomponents of FIG. 1 are depicted as a system, one or more of thecomponents depicted may be removed, depending on the particularembodiment. The system 10 also permits a consumer to access his/herconsultation data from a plurality of computing devices in disparatelocations at the time and place of his/her choosing.

As illustrated, the system includes a network 100, which may include alocal area network and/or a wide area network, such as the internet, apublic switched telephone network (PSTN), a mobile telephone network,etc. any of which may be configured to provide a wired and/or wirelesscommunication platform. Coupled to the network 100 is a kiosk 102 orother terminal, which may be configured for providing a personalconsultation, a semi-personal consultation, and/or a virtualconsultation. Similarly, the kiosk 102 may include an outer shell and/ora dispensing unit 105 for dispensing a calibration device 110 and/or oneor more cosmetic products to a consumer before, during or after apersonal consultation.

A personal consultation may include a human consultant that isphysically present at the kiosk 102 to provide an analysis, treatmentrecommendations, and/or product recommendation. The recommendations maybe provided by a user interface, a printable page, an email, a textmessage (e.g., SMS), and/or via other protocols.

Similarly, a semi-personal consultation may include utilizing an imagecapture device 104 a, a display device 106, and/or a communicationsdevice 108 to communicate with a human consultant that is remotelylocated from the kiosk 102, such as via a consultation center 112. Asillustrated, the image capture device 104 a (such as described in U.S.Pat. Nos. 7,965,310, 7,719,570), display device 106, and/orcommunications device 108 may be built into a housing of the kiosk 102.The kiosk 102 may further comprise the dispensing unit 105 for storingone or more cosmetic products that may be purchased and/or dispensed toa consumer from the kiosk 102. In one embodiment, the image capturedevice 104 a and the display device 106 are positioned in the dispensingunit 105 on a side panel of the kiosk 102. The image capture device 104a may be movably mounted within the housing 109 so that it can bepositioned in a configuration suitable to capture the consumer's faceduring use.

For example, the image capture device 104 a may be pivotally mounted ina bracket attached to the kiosk 102, wherein an arm may be used by theconsumer to pivot the image capture device 104 a. Similarly, the imagecapture device 104 a may be slidably mounted on one or more verticaltracks that permit sliding movement of the image capture device 104 awithin the kiosk 102. Additionally, a call button or other userinterface may be positioned on the kiosk 102 for automaticallyinitiating an audio/video consultation with the consultation center 112.The call button (or other user interface) may be connected tocommunication hardware and/or software within the kiosk 102 forfacilitating a communication with the consultation center 112.

A variety of mechanisms (not shown) may be used for dispensing thecosmetic products from the dispensing unit 105 of the kiosk 102. Somemechanisms that may be suitable for use are described in U.S. Pat. No.6,758,370; 2009/0306819; 2010/0025418; and 2010/0138037. In oneembodiment, one or more cosmetic products may be selected/purchased by aconsumer from the kiosk 102 following a personal consultation in whichthat cosmetic product was recommended to the consumer.

The semi-personal consultation may provide similar analysis and/orrecommendations, which may similarly be provided via a user interface, aprintable page, an email, a text message (e.g., SMS), and/or otherprotocols. The virtual consultation may be a consultation that isprovided by a virtual consultant (e.g., a computer program). Dependingon the particular embodiment, the virtual consultation may utilize theimage capture device 104 a, the display device 106, a calibration device110, and/or the communications device 108. The virtual consultation mayprovide a similar analysis and/or recommendations as above, which maysimilarly be provided via a user interface, a printable page, an email,a text message (e.g., SMS), and/or other protocols. The user may utilizethe calibration device 110 to facilitate adjustment of color settingsand/or resolutions of images captured by the image capture device 104 a(or other image capture device) to a predetermined image standard. Thismay provide the ability for consistent analysis of images, regardless ofthe current lighting characteristics, image capture device 104 acharacteristics, orientation of the consumer, etc.

In some embodiments, the kiosk 102 may be located at a retail store, amedical office, a mall, a public venue, and/or other location foranalyzing and recommending cosmetic products and/or other products.However, persons of skill in the art will readily appreciate that thekiosk 102 may be used anywhere without departing from the scope andspirit of the present disclosure. For example, the kiosk 102 could beused in a doctor's office for diagnostic purposes and archiving patientdata. The kiosk 102 may include the image capture device 104 a, whichmay be configured with computing capabilities for acquiring images to beanalyzed. In some instances, the kiosk 102 will be located remotely fromconsultation center 112, the user computing device 118, the foreignremote computing device 116, and the native remote computing device 114.For example, the devices, kiosk 102, and consultation center 112 mightbe located in different buildings, different cities, different states,or different countries.

More specifically, the image capture device 104 a may includepositioning equipment, lights, and a digital image generator such as adigital camera, an analog camera connected to a digitizing circuit, ascanner, a video camera, etc. The components of the image capture device104 a may be arranged at predetermined distances and at predeterminedangles relative to one another to maximize the quality of the acquiredimage. For example, a positioning device for stabilizing the face of aperson may include a chin rest and/or a forehead rest. In someembodiments, a digital image generator may be placed at a predetermineddistance and at a predetermined angle relative to the positioningdevice.

While any digital or analog image capture device may be utilized, insome embodiments, an electromagnetic capture device may be utilized. Insome embodiments, a High Definition camera with image capture and videocapability may be utilized. The camera may contain features such as autofocus, optics and sensors. Specifically, the user may self-align thetarget substrate to be captured and/or measured for analysis. The targetsubstrate may include a portion of a user's body, which may be anon-homogeneous/homogeneous shiny or matte substrate. In someembodiments, the target substrate comprises a consumer's face, skin,hair, etc.

The calibration device 110 may also be aligned with the image capturedevice 104 a, at which point the user may trigger an electromagneticmeasurement from the kiosk 102. In response, electromagnetic waves maybe captured from the target substrate. Digital data may be determinedfrom the captured electromagnetic waves. Based on the digital data, theuser may be given an analysis, a treatment recommendation, and/or aproduct recommendation, which optionally could be purchased and/ordispensed from the kiosk 102. Items can also be ordered via a graphicaluser interface or call center agent and shipped to the consumer.

The image capture device 104 a may also be configured to generate colordata from the target substrate and one or more calibration devices (suchas the calibration device 110), potentially in conjunction with asource, such as a xenon flash lamp, a linear flash, a ring flash orother light sources. The image capture device 104 a may include chargecoupled devices (CCDs), complementary metal oxide semiconductor (CMOS)devices, junction field effect transistor (JFETs) devices, linear photodiode arrays or other photo-electronic sensing devices. As also notedabove, the target substrate may take any of a number of forms, includingfor example the skin, eyes or teeth of the user of the kiosk 102. Thecalibration device(s) may be stored within the kiosk 102 and dispensedthere from and/or otherwise provided to the user and may include asample with one or more regions whose light intensity characteristicsare known.

As also illustrated in FIG. 1 and described in more detail below, thesystem may include the calibration device 110, which may be used incombination with the kiosk 102 and/or the user computing device 118. Thecalibration device 110 is easily portable, thereby enabling its use withthe various devices that may be located remotely from each other. Asdiscussed more fully hereafter, the calibration device 110 permitsconsultations and/or analysis of skin features using a variety of imagecapture devices connected to a variety of different computing/mobiledevices in various locations while providing a more consistent image tothe consumer and minimizing image variability due to differences inhardware and lighting. The calibration device may be dispensed from thekiosk 102, may be packaged with a cosmetic product and sold with thecosmetic product. The cosmetic product may be delivered to a consumer atresidential location, such as for example a home where the usercomputing device is located, or may be distributed from a retaillocation.

The kiosk 102 is also connected to one or more output devices such asthe display device 106, a printer, etc. The display device 106 mayinclude a cathode ray tube (CRT), liquid crystal display (LCD), or anyother type of display. Similarly, the display device 106 may beconfigured as a touch screen integrated with a video screen. The displaydevice 106 may be configured to generate images, which may includeoperator prompts, preferences, options, and digital images of skin. Theprinter may include a laser printer, ink jet printer, or any other typeof printer. The printer may be used to print out digital images and/oranalysis results for the analyzed person.

According to some embodiments, the kiosk 102 may also include anelectromagnetic source and a plurality of filters in a predeterminedarrangement to be used in measuring an electromagnetic radiationresponse property associated with the target substrate. In such anembodiment, at least a portion of the waves generated by the source maybe captured after the waves pass through a first polarized filter,reflect from the user, and pass through a second polarized filterarranged in a cross polar arrangement with respect to the firstpolarized filter. Additionally, the kiosk 102 may be configured tocapture electromagnetic waves that pass through an attenuating filterand reflect from the one or more calibration devices. In such acircumstance, the digital data obtained may be used to calibrate and/orrecalibrate the apparatus.

The kiosk 102 may additionally include a controller (and/or processor),which may include one or more processing units operatively coupled toone or more memory devices and one or more interface circuits (similarto that depicted for the native remote computing device 114 in FIG. 2A).In turn, the one or more interface circuits may be operatively coupledto one or more input devices, one or more output devices, anelectromagnetic source and an electromagnetic capture device.

The one or more processing units may be of a variety of types, forexample including microprocessors, microcontrollers, digital signalprocessors, specialized mathematical processors, etc. The memorydevice(s) may include volatile memory and/or non-volatile memory, andmay be in the form of internal and/or external memory (e.g., flashcards, memory sticks, etc.). The memory device(s) may store one or moreprograms that control the function of the kiosk 102. The memorydevice(s) may also store data indicative of screen displays, bit maps,user instructions, personal identification information, demographicdata, digitized images, color data, light intensity data, histogramdata, and/or other data used by the apparatus and/or collected by theapparatus. The interface circuit may implement any of a variety ofstandards, such as Ethernet, universal serial bus (USB), and/or one ormore proprietary standards.

The one or more input devices may be used to receive data, signals,identification information, commands, and/or other information from theuser of the kiosk 102. For example, the one or more input devices mayinclude one or more keys or buttons, a voice or gesture recognitionsystem and/or a touch screen. The one or more output devices may be usedto display or convey prompts, instructions, data, recommendations and/orother information to the user of the kiosk 102. For example, the one ormore output devices may include the display device 106, other displaydevices, lights, and/or speakers. Additionally, depending on theparticular embodiment, the kiosk 102 may be configured as auser-operated mobile device or system.

Additionally, the system 10 may include a consultation center 112located remotely from the kiosk 102 and/or user computing device 118.The consultation center 112 may be coupled to the kiosk 102 and/or theuser computing device 118, such that a user may conduct a semi-personalconsultation with a consultant that is located at the consultationcenter 112. As an example, the user may access a user interface (such asthose described below) to select a consultant, who will then becontacted utilizing the communications device 108. The consultationcenter 112 may include a plurality of audio, video, and/or datacommunication hardware and software and may receive (or initiate) thecall to begin the consultation with the user. The consultant may controlat least a portion of the functionality of the kiosk 102 and/or usercomputing device 128 to remotely capture images, dispense thecalibration device 110, and/or perform other functions.

Depending on the particular embodiment, user interfaces and/or otherdata may be provided by the kiosk 102, user computing device 118, and/ora native remote computing device 114. The native remote computing device114 may include a memory component 140, which stores receiving logic 144a, analysis logic 144 b, and/or other logic for facilitating performanceof the consultation. With this logic, the native remote computing device114 may send user interface data to the kiosk 102 and/or user computingdevice 118. Additionally, the native remote computing device 114 maydetermine whether the consultation is a personal consultation,semi-personal consultation, and/or virtual consultation. If theconsultation is a semi-personal consultation, the native remotecomputing device 114 may interact with the consultation center 112 tofacilitate the consultation. If the consultation is a virtualconsultation, the native remote computing device 114 may perform theconsultation analysis and/or perform other functions. More specifically,while in some embodiments, the kiosk 102 and/or user computing device118 may include logic and/or hardware for providing user interfaces,performing the analysis, and providing treatment and productrecommendations, in some embodiments, the native remote computing device114 may provide this functionality.

Depending on whether the consultation is a personal consultation, asemi-personal consultation, and/or a virtual consultation, the nativeremote computing device 114 may access the consultation center 112, asdescribed above with regard to the kiosk 102. Additionally, the nativeremote computing device 114 and/or the user computing device 118 mayaccess the foreign remote computing device 116 to retrieve data from aprevious consultation, share the consultation with friends and/orperform other functions.

Also included is a foreign remote computing device 116. The foreignremote computing device 116 may be configured as a computing device forstoring data. As an example, the foreign remote computing device 116 maybe configured as a social network server, a storage server, a usercomputing device, a consultation server, and/or other device forperforming the described functionality. As such, the kiosk 102 and/oruser computing device 118 may prompt the user to save the data from theconsultation to the foreign remote computing device 116 for subsequentretrieval by other devices, such as the kiosk 102, the native remotecomputing device, and/or the user computing device 118. If the user isprompted to save the data on the foreign remote computing device 116,such as a social network, the kiosk 102 may have a dedicated profilepage on the social network to upload the consultation data. Theconsultation data may be tagged for the user, so that the data is alsoincluded in the user's profile page. In order to protect privacy, thedata may be redacted on any public posting of the data and/or providedso that only the user may access the data. Similarly, in someembodiments, the kiosk 102 and/or the user computing device 118 canupload the consultation data directly to the user's profile page. Insuch a scenario, the kiosk 102 and/or the user computing device 118 mayreceive the user's login information.

As an example, if the user is performing a skin treatment, an image ofthe user's face may be captured. After the consultation, the user maywish to store the image, so that the user can compare this image withpast and/or future images. As such, the kiosk 102, the user computingdevice 118, the native remote computing device 114, and/or theconsultation center 112 may facilitate sending and/or storing this data.Depending on the particular embodiment, the data may be sent to thenative remote computing device 114, the foreign remote computing device116, and/or a user computing device 118.

Also included in the embodiment of FIG. 1 is the user computing device118. The user computing device 118 may include a personal computer,notebook, mobile phone, smart phone, laptop, tablet, and/or other devicefor communicating with other devices on the network 100. The usercomputing device 118 may incorporate an image capture device 104 b thatis different from the image capture device 104 a utilized by the kiosk102. The image capture device 104 b may include some, if not all thesame hardware, software, and/or functionality as described above withrespect to the image capture device 104 a. The image capture device 104b and may be utilized with the user computing device 118 as astand-alone device that is connectable to the user computing device 118by a cable (e.g., a USB cable or video cable), and/or may be integral toor otherwise hard wired to the user computing device 118.

Additionally, the user computing device 118 may be configured as ageneral purpose computer and/or may take the form of a personalcomputer, a mobile phone, a smart phone, a tablet, a laptop, and/orother type of computing device. The user computing device 118 mayprovide similar functionality as the kiosk 102 and thus permit the userto send a request for and receive consultations at a wide variety oflocations. As such, the user computing device 118 may be configured tolog into the native remote computing device 114 to perform analysis.More specifically, the user may access the native remote computingdevice 114 to perform a consultation or to complete a previouslyinitiated consultation.

Example Devices

FIG. 2A depicts the native remote computing device 114 that may beutilized for providing a consultation, according to embodimentsdisclosed herein. Depending on the particular embodiment, the nativeremote computing device 114 (and/or other computing devices depicted inFIG. 1) may be configured as a general purpose computer programmed toimplement the functionality described herein. Similarly, the nativeremote computing device 114 may be an application-specific devicedesigned to implement the functionality described herein. In theillustrated embodiment, the native remote computing device 114 includesa processor 230, input/output hardware 232, network interface hardware234, a data storage component 236 (which stores user data 238 a, productdata 238 b, and/or other data), and the memory component 140. The memorycomponent 140 may be configured as volatile and/or nonvolatile memoryand as such, may include random access memory (including SRAM, DRAM,and/or other types of RAM), flash memory, secure digital (SD) memory,registers, compact discs (CD), digital versatile discs (DVD), and/orother types of non-transitory computer-readable mediums. Depending onthe particular embodiment, the non-transitory computer-readable mediummay reside within the native remote computing device 114 and/or externalto the native remote computing device 114.

Additionally, the memory component 140 may store operating logic 242,the receiving logic 144 a, and the analysis logic 144 b. The receivinglogic 144 a and the analysis logic 144 b may each include a plurality ofdifferent pieces of logic, each of which may be embodied as a computerprogram, firmware, and/or hardware, as an example. A local communicationinterface 246 is also included in FIG. 2A and may be implemented as abus or other communication interface to facilitate communication amongthe components of the native remote computing device 114.

The processor 230 may include any processing component operable toreceive and execute instructions (such as from the data storagecomponent 236 and/or the memory component 140). The input/outputhardware 232 may include and/or be configured to interface with amonitor, positioning system, keyboard, touch screen, mouse, printer,image capture device, microphone, speaker, gyroscope, compass, keycontroller, and/or other device for receiving, sending, and/orpresenting data. The network interface hardware 234 may include and/orbe configured for communicating with any wired or wireless networkinghardware, including an antenna, a modem, LAN port, wireless fidelity(Wi-Fi) card, Bluetooth™ hardware, WiMax card, mobile communicationshardware, router(s) and/or other hardware for communicating with othernetworks and/or devices. From this connection, communication may befacilitated between the native remote computing device 114 and/or othercomputing devices.

The operating logic 242 may include an operating system and/or othersoftware for managing components of the native remote computing device114. Other functionality is also included and described in more detail,below.

It should be understood that the components illustrated in FIG. 2A aremerely exemplary and are not intended to limit the scope of thisdisclosure. While the components in FIG. 2A are illustrated as residingwithin the native remote computing device 114, this is merely anexample. In some embodiments, one or more of the components may resideexternal to the native remote computing device 114. It should also beunderstood that while the native remote computing device 114 in FIG. 2Ais illustrated as a single device; this is also merely an example. Insome embodiments, the receiving logic 144 a and/or the analysis logic144 b may reside on different devices. Additionally, while the nativeremote computing device 114 is illustrated with the receiving logic 144a and the analysis logic 144 b as separate logical components, this isalso an example. In some embodiments, a single piece of logic may causethe native remote computing device 114 to provide the describedfunctionality.

FIG. 2B depicts one embodiment of a calibration device 110 that may beutilized for providing a color calibration of images for a consultation.The calibration device 110 may be configured as a headband formed from aflexible, elongate strip of material 259 having a first end 253 a, asecond end 253 b, and one or more slits 253 c adjacent the second end253 b for receiving the first end 253 a. In some embodiments, the stripof material 259 has a width W from about 0.5 cm to about 6 cm and alength L from about 40 cm to about 100 cm. In some embodiments, theheadband has a width W from about 2 cm to about 4 cm and a length L fromabout 60 cm to about 80 cm. By inserting the first end 253 a into theslit 253 c, the strip of material 259 can be formed into an adjustablering or headband that may be sized to accommodate various users.

The headband includes alignment indicator 252, at least one colorcorrection region 256 (which may include a first color correction region256, a second color correction region 256, etc.), an identifier 255, ameasurement component 257, at least one resolution indicator 254, and/orother components. The alignment indicator 252 may generally bifurcatethe headband. In one embodiment, the alignment indicator 252 is providedin the form of a substantially vertical stripe, which a user of theheadband can align with the mid-point of their face during use. Thealignment indicator 252 can be provided in a wide range of sizes,shapes, and colors. In some embodiments, the color correction region 256includes a plurality of color chips 258 having at least onepredetermined chip color, each chip color being a different color.

Additionally, the measurement component 257, and/or resolution indicator254 (which may be embodied as green cross shapes on either side of thealignment indicator 252) are arranged on opposite sides of the alignmentindicator 252. In some embodiments, the color correction region 256, themeasurement component 257, and/or resolution indicator 254 aresymmetrically arranged on opposite sides of the alignment indicator 252.The calibration device 110 may be made from a wide variety of materials,including paper, cardboard, and plastics.

Additionally, in some embodiments the calibration device 110 may beconstructed of a predetermined background color that is predicted to beabsent from the target substrate. Similarly, in some embodiments, thebackground color is selected based on a predicted contrast in the LABcolor space. This color may include blue, green, and/or other colors.

During the consultation at the kiosk 102 and/or at the user computingdevice 118, the user may wear the calibration device 110 on his/her headwith the alignment indicator 252 aligned near the mid-point of thesubject's face or subject's forehead so that color correction regions256 are positioned on either side of the face. In many instances, it maybe desirable to capture an image of the face from an oblique anglethereto (such as at a 45 degree angle), as seen for example in FIG. 8.In these instances, having a color correction area positioned on eitherside of the mid-point of the face permits an image capture at anyoblique angle on either side of the face without having to repositionthe headband so that a color correction region 256, resolution indicator254, and/or measurement component 257 are within the field of thecaptured image.

The image may be captured at an oblique angle to more provide visualaccess to the cheek and eye area of the target substrate with a reducedeffect from shadows and distortion. In some embodiments, the cheek andeye areas of the face provide a suitable area for wrinkle and spotdetection, the oblique angle may be utilized.

Generally speaking, the user's hair may be pulled back or otherwisearranged so that it does not cover the color correction region 256, theresolution indicator 254, and/or measurement component 257, and theuser's head should be rotated from about 30 degrees to about 60 degreesrelative to the image capture device at the time of taking the image(e.g., somewhere between a profile or side image and an image takenlooking directly into the image capture device).

As discussed in more detail below, the components of the calibrationdevice 110 may be utilized for the kiosk 102, user computing device 118,and/or native remote computing device 114 to adjust imagery to provide aconsistent analysis of the target substrate.

As illustrated, the color correction regions 256 may include a pluralityof color chips in the form of squares (or other shapes) of varyingcolors. Additionally, the alignment indicator 252 and/or other markingson the calibration device 110 may be different colors.

While each of the plurality of color chips of the color correctionregions 256 are illustrated as being square in shape; this is not arequirement of the calibration device 110. Rectangular areas and/orother shapes may be utilized so as to be used in the calibration device110 of the present disclosure. According to certain embodiments, thecalibration device 110 may also include instructions disposed thereon.Depending on the particular embodiment, the instructions may be disposedon a backside of the calibration device 110. As described more fullyhereafter, in use, the known colors on the headband may be compared tocolor values captured in an image of the headband during a consultation.The facial image can then be color corrected based upon the differencebetween the known color values of the color correction region 256 of thecalibration device 110 and the color values of the color correctionregion 256 captured in an image by the image capture device 104 a, 104b. Use of the same type of calibration device 110 with multipledifferent image capture devices 104 a, 104 b provides a common colorcalibration standard that enables a user's image to be captured by avariety of different image capture devices and then color corrected sothat the corrected image most closely resembles the true colors of theuser's face regardless of what type of image capture device is used.This further enables more meaningful tracking of user's experience overtime with one or more cosmetic products using one or more devices.

FIG. 2C depicts a product package 298, which may include a calibrationdevice 110 and a cosmetic product 299, according to some embodimentsdisclosed herein. As illustrated, the calibration device 110 may beincluded as part of a kit. The product package 298 may also include acosmetic product 299, which may be specifically designed to treat acondition of a subject. However, in some embodiments, the product 299may simply be a general treatment product that applies to all subjects.Depending on the particular embodiment, the calibration device 110 maybe included inside the product package 298 and/or may be adhered to awall of the product package 298.

Example User interfaces

FIG. 3 depicts one embodiment of a user interface 360 for providing aproduct and/or treatment consultation, such as in the computingenvironment from FIG. 1. As illustrated, the user interface 360 includesa video area 362, a “new users” option 364, and a “submit” option 366.The video area 362 may provide the user with one or more instructionalvideos for performing an analysis as described herein. If the user isnew to the service, the user may select the new users option 364, whichprovides additional options for creating an account, such as thosedepicted in FIG. 4. If the user is a returning user, the user may enterthe requested account information and select the submit option 366.Additional options may also be provided, such as an “articles” option, a“glossary” option, an “FAQ” option, a “feedback” option, and/or otheroptions.

As discussed above, depending on the particular embodiment, the userinterface 360 (and/or other user interfaces described herein) may beprovided at the kiosk 102 and/or at the user computing device 118. Itwill be appreciated that a wide variety of user interface features,functionality, options, and layouts can be provided, deleted, or alteredfrom what is shown and described in FIGS. 3-15.

FIG. 4 depicts a user interface 460 for creating a profile forperforming a product and/or treatment consultation, according to someembodiments disclosed herein. As illustrated, the user interface 460 maybe accessed in response to selecting the “new users” option 364 fromFIG. 3. More specifically, upon selecting the “new users” option 364,the user may be prompted to create a profile. As part of creating aprofile, the user may be asked for the month and year of birth in areas462 and 464. The user may be asked his/her gender in area 466, his/herethnicity in area 468, and his/her skin type in area 470. Additionalquestions, such as contact information, billing information, etc. may beasked of the user in response to selection of the additional questionsoption 472.

It should be understood that the data in FIG. 4 is collected tocategorize the subject into a subject type. More specifically, subjectsin certain geographic areas may receive similar types and amounts ofsunlight. Similarly, subjects of similar ages, skin types, andethnicities may additionally be expected to have similar skin, hair,lip, etc. characteristics. Further, once the subject is analyzed asdescribed herein, the subject's information may be stored for comparisonwith other subjects.

FIG. 5 depicts one embodiment of a user interface 560 for selecting aconsultant. As illustrated, the user may select one of a plurality ofdifferent consultants, such as via selection of an option 562 a-562 f.Additionally, as discussed above, the consultants may be a live person(e.g. physically at the kiosk 102 or user computing device 118), aremote person (e.g., accessible via the consultation center 112), and/ora virtual person (e.g., software being executed to simulate a realperson). Depending on the embodiment, an indication of the type ofconsultation may be provided in the user interface 560.

FIG. 6 depicts one embodiment of a user interface 660 for viewing aprofile of a selected consultant, according to embodiments disclosedherein. As illustrated, in response to selection of the option 562 cfrom FIG. 5, the selected consultant may be contacted for aconsultation. If the selected consultant is physically present at thekiosk 102 and/or user computing device 118, the consultant may becontacted and the consultation may be performed accordingly in person.If the selected consultant is remotely located, the kiosk 102 and/oruser computing device 118 may contact (or be contacted by) theconsultant via the consultation center 112.

As such, if the selected consultant is a remotely located person or avirtual consultant, the user interface 1150 may be provided to include avideo communication stream to conduct the consultation.

FIG. 7 depicts one embodiment of a user interface 760 for conducting aconsultation. As illustrated, in response to creating a profile and/orlogging into the service, the user interface 760 may be provided. Morespecifically, the user interface 760 may include a day calendar 762 forthe user to select a day for the consultation, as well as a timecalendar 764 for selecting a time for the consultation. By selecting aschedule option 766, the consultation may be scheduled. When the timefor the consultation approaches, the user may be contacted for areminder via for example, telephone call, email, text message, etc.

FIG. 8 depicts one embodiment of a user interface 860 for uploadingimagery to perform a consultation. As illustrated, the user may capturean image using the image capture device 104 a, 104 b and facilitateupload of that image for use in the consultation. Depending on theparticular image capture device, the lighting, and/or other factors, theimage may differ from other images that may be uploaded. Accordingly,the native remote computing device 114 (and/or other computing devicedescribed herein) may alter the uploaded image to predeterminedcharacteristics. This image 862 may then be provided to the user. Alsoincluded in the user interface 860 are a “perform spot analysis” option864, a “perform wrinkle analysis” option 866, a “view past results”option 868, a “print results” option 870, a “chat with a consultant”option 872, and a “connect with social media” option 874.

In response to selection of the perform spot analysis option 864, thenative remote computing device 114 and/or other computing device mayperform an analysis of the image 862 and may identify facialcharacteristics, such as spots. Similarly, upon selection of the performwrinkle analysis option 866, wrinkles and/or other facialcharacteristics may be determined from the image 862. In response toselection of the view past results option 868, images, analysis, productrecommendations, and/or other data related to previous consultations maybe provided.

Additionally, in response to selection of the print results option 870,the user interface 860 may be sent to a printer. In response toselection of the chat with a consultant option 872, a consultant (suchas those depicted in FIG. 5) may be connected to the user forconsultation. In response to selection of the connect with social mediaoption 874, the images and/or data may be sent to a social network forposting and/or storage. Other posting and storage options may also beprovided.

FIG. 9 depicts a user interface for identifying wrinkles on the targetsubstrate, according to some embodiments disclosed herein. Asillustrated, the user interface 960 may be configured to display animage 962 of the user that has been altered from the version captured bythe image capture device 104 a, 104 b. While the captured image mayinclude the target substrate of the user (e.g., the right or left cheekand around the eye) and the calibration device 110, the image 962 may beadjusted to provide only the target substrate. Additionally, other imageprocessing may be performed, which may include a color correction,spatial image adjustment, and facial characteristic analysis to identifywrinkles and/or other facial characteristics, which have been markedwith indicators 964.

Referring again to FIG. 9, the user interface 960 may also include a“perform spot analysis” option 966, a “view image” option 968, a “viewpast results” option 974, a “chat with a consultant” option 976, and a“connect with social media” option 978. The user interface 960 alsoincludes an analysis scale 972 for indicating to the user how healthylooking the target substrate is. The user interface 960 also provides arecommended product 980, as well as a “purchase” option 982 to purchasethe recommended product.

The perform spot analysis option 966 may provide the user with adifferent analysis of the uploaded image, as described with regard toFIG. 10. The view image option 968 may provide the user with theoriginal uploaded image. The view past results option 970 may providethe user with results from previous consultations, which may have beenstored by the native remote computing device 114, the foreign remotecomputing device 116, the kiosk 102, the user computing device 118,and/or the consultation center 112. The print results option 974 mayprint the image, analysis, and/or other data of the analysis provided inthe user interface 960. The chat with a consultant option 976 mayprovide the user with access to one or more consultants, as discussedabove. The connect with social media option 978 may provide the userwith options for storing and/or retrieving data from a social network.The recommended product 980 may be determined based on the currentand/or past analysis, as well as the age, skin type, allergies, zipcode, etc. of the subject. The buy now option 982 may allow the user topurchase the recommended product and/or view other products directly.Additionally, the recommended product 980 may be sold and/or providedwith the calibration device 110 in the form of a kit.

It should also be understood that in some embodiments, the user maydesire a change to his/her appearance and/or health. In suchembodiments, a user interface may be provided to the user for indicatingthe change that the user wishes to make. As an example, if the subjectwishes to change his/her hair color, the user may indicate this changein one of the user interfaces. In response, the kiosk 102, usercomputing device 118, consultation center 112, native remote computingdevice 114, and/or foreign remote computing device 116 can provide theuser with a pallet of images of the subject with the different haircolors that are possible with the subject's target substrate. The usermay then select the desired color and the products and/or treatment maybe provided for creating the desired result.

It should also be understood that while the user interfaces depictedherein may be configured to facilitate the communication of data amongthe native remote computing device 114, the consultation center 112, thekiosk 102, the foreign remote computing device 116, and/or the usercomputing device 118. More specifically, in some embodiments, the usercomputing device 118 may receive user input, which is sent to the nativeremote computing device 114. The native remote computing device 114 maysend the data to the consultation center 112 for viewing by theconsultant. The consultant may then discuss the results with the user.

FIG. 10 depicts a user interface 1060 for identifying spots on thetarget substrate, according to some embodiments disclosed herein. Asillustrated, once the image of the user has been altered forconsistency, an analysis may be performed to determine at least onefacial characteristic and other issues in the target substrate. Morespecifically, in FIG. 10, the image 1062 (a non-limiting example of analtered image) may indicate the areas of spots with indicators 1064 onthe user's face.

Also included in the user interface 1060 are a “perform wrinkleanalysis” option 1066, a “view image” option 1068, a “view past results”option 1070, an analysis scale 1072, a “print results” option 1074, a“chat with a consultant” option 1076, and a “connect with social media”option 1078. The perform wrinkle analysis option 1066 may be configuredto send the user back to the user interface 960. The view image option1068 may provide the user with an unaltered version of the image 1062.The view past results option 1070 may provide the user with analysisand/or other data from previous consultations. The print results option1074 may send at least a portion of the data from the user interface1060 to a printer. The chat with a consultant option 1076 may place theuser in contact with a consultant, as discussed above. The connectionwith social media option 1078 may allow the user to save, post, upload,and/or perform other interactions with a social network.

Also included is a recommended product 1080, which has been determinedas being beneficial based on the identified facial characteristics andthe user's other information. A “purchase” option 1082 may provide theuser with the ability to immediately purchase the recommended product1080.

It should be understood that while spots are the depicted defect in FIG.10, this is merely an example. More specifically, any type of conditionmay be identified, depending on the particular embodiments. Examples forskin include moles, freckles, pores, wrinkles, spots, etc. Examples forhair include split ends, gray hair, thinning hair, etc. Some methodssuitable for use with the present invention for analyzing skin imagesare described in U.S. Pat. No. 6,571,003.

It should also be understood that, as illustrated in FIGS. 9 and 10, insome embodiments, an altered image that is based on the originaluploaded image and the location of the defect areas may be provided. Thealtered image visually identifies the plurality of defect areas locatedin the uploaded image by electronically altering the color of aplurality of pixels substantially in the area containing the skin defect(e.g., on or around the defect area) to at least one color visuallydistinct from the skin color of the uploaded image. For example, theskin color of each pixel in the defect area may be shifted to a shade ofblue to create a transparent overlay. In another example, a circle couldbe drawn around each of the facial characteristics to visually identifythe location of the spots. Other alterations may also be provided.

Additionally, a numerical severity may be associated with the defectareas. In one embodiment, a color content associated with the defectarea may be subtracted from the color content of the area immediatelysurrounding the defect area. For example, if the pixels used to create ared spot have a red content of 60% and the pixels used to create thesurrounding skin color have a red content of 10%, then the numericalseverity associated with the red spot defect in this example may bedetermined to be 50. Similarly, in some embodiments, the number ofgeometric coordinates necessary to cover the defect area is thenumerical severity. For example, if a detected pore covers 30 pixels,then the numerical severity associated with that pore may be determinedto be 30. The severity of multiple instances of a particular defect typemay be aggregated. For example, multiple severities may be summed oraveraged.

Further, the aggregated severity may be normalized, based on humanperception coefficients. For example, if it is determined in a clinicalstudy that red spots are twice as noticeable as brown spots, theaggregated severity associated with the red spot analysis may bedoubled. Alternatively, in this example, the aggregated brown spotseverity may be halved. Of course, a person of skill in the art willreadily appreciate that more than two defect types may be normalized.

A percentile for the normalized severity may additionally be determinedusing data associated with a certain population of people. Thepopulation data used may be specific to the analyzed person's age,geographic location, ethnic origin, or any other factor. For example, if55% of a sample group of people in the analyzed person's age group had anormalized severity for the current defect type below the analyzedperson's severity, and 45% of the sample group had a severity above theanalyzed person's severity, then a percentile of 55 or 56 is determined.

When there are no more defect types to process, an overall skin severityand an overall percentile may be calculated. The overall skin severitymay be an aggregation of the plurality of individual skin defectseverities. For example, the severities determined by each defect may besummed or averaged. The overall percentile may be calculated asdescribed above for the individual skin defect percentiles; however, adifferent data set representing overall severities of a population ofpeople may be used. Again, the population data may be selected based onthe analyzed person's demographics.

In addition to an overall skin severity based on the aggregation ofindividual skin defect severities, one or more overall skincharacteristics may be determined. An overall skin characteristic maynot depend on the detection of any individual skin defects. For example,an overall smoothness/roughness magnitude may be determined. Such adetermination may include certain skin defects (e.g., analyze entireimage or sub-image) or it may exclude certain skin defects (e.g., do notanalyze pixels in the hyper-pigmentation defect areas).

It should also be understood that while the examples depicted in FIGS. 9and 10 illustrate analysis of the image as a whole, in some embodimentssub-images may be determined. A sub-image is a portion of the originallyacquired image upon which analysis will be performed. By eliminating aportion of the acquired image from the analysis process, fewer errorsoccur. For example, by excluding consideration of the eyes and nose fromthe analysis process, an incorrect determination that a largediscoloration of the skin is present is avoided.

As an example, a decision may be made to use automatic or manualsub-image determination. In one embodiment, this decision is made by theuser. However, in some embodiments the selection may be automaticallydetermined. In such an instance, the native remote computing device 114(and/or other computing device) may analyze or partially analyze theimage automatically, and based on the results of that analysis, adecision is made regarding whether to use automatic or manual sub-imagedetermination. For example, if the automatic sub-image determinationincludes a result indicative of a confidence level (e.g., how sure isthat a nose has been found), and that confidence result is below somepredetermined threshold, then a manual sub-image determination may beperformed.

If a manual sub-image determination is selected, a decision may be madeto use prompted or unprompted sub-image determination. This decision maybe made by the user. If unprompted sub-image determination is selected,the operator draws a virtual border for the sub-image. If promptedsub-image determination is selected, the native remote computing device114 and/or other computing device prompts the user to select a series oflandmarks on the displayed image (e.g., corner of the mouth, then cornerof the nose, then corner of the eye, etc.). Subsequently, the nativeremote computing device 114 and/or other computing device may draw inthe sub-image border by connecting the landmarks.

If automatic sub-image determination is selected, a decision is made touse fully automatic or semi-automatic sub-image determination. Ifsemi-automatic sub-image determination is selected, the user may selectseveral landmarks, but not all of the landmarks, for the sub-image. Thenative remote computing device 114 then may determine the remaininglandmarks automatically by comparing the user entered landmarks to apredetermined landmark template (e.g., a standard mask) andinterpolating the user entered landmarks using shape warping algorithms.

Similarly, as an example, the remaining landmarks may be calculated bytaking the spatial difference vector (delta x, delta y) between the userentered landmarks and a standard mask for each of the user enteredlandmarks. Then, the remaining landmarks may be calculated using abilinear interpolation of the spatial difference vectors and the x, ycoordinates of the two closet user entered landmarks. Subsequently, thenative remote computing device 114 may draw in the sub-image border byconnecting the landmarks (both user entered landmarks and automaticallydetermined landmarks).

If fully automatic sub-image determination is selected, the nativeremote computing device 114 and/or other computing device determines allof the landmarks for the sub-image automatically by searching forpatterns in the digital image indicative of predetermined landmarks.Once the main sub-image is determined, additional sub-images may bedetermined. In one embodiment, an arc is drawn by the native remotecomputing device 114 between two of the landmarks to define an “undereye” sub-image border. The user may then adjust the size of the “undereye” sub-image. In some embodiments, a sub-image is electronicallydetermined by comparing a plurality of color values of a plurality ofpixels to a predetermined threshold indicative of skin color.

Once the sub-images are determined, the sub-images may be analyzed tolocate defect areas and compare the severity of the defect areas to anaverage skin severity of a population of people. In one embodiment,defect areas are areas in the sub-image which meet certain criteria(e.g., a red spot). The severity of a particular instance of a defect isan estimation of the degree to which humans perceive one defect as being“worse” than another. For example, a large red spot is considered moresevere than a small red spot. Many different defect types may belocated. For example, skin elasticity features such as wrinkles and/orfine lines may be located. Skin smoothness, skin texture, follicularpores, inflamed red spots such as acne, hyperpigmented spots such assenile lentigenes, nevi, freckles, as well as many other skin defectsmay also be located using a variety of known algorithms.

Additionally, an index variable may be initialized to zero. The indexvariable may be utilized to keep track of which type of skin defect isbeing analyzed. If only one defect type is being analyzed, the indexvariable may be eliminated. A plurality of areas in the sub-image maycontain the current defect type are located. For example, if thesub-image contains six red spots (as defined by a known red spotdetection algorithm) then six locations in the sub-image are determined.Each location may be identified using a single set of geometriccoordinates specifying the approximate center of the located defect, or,each location may be identified by a set of geographic coordinatescovering a region affected by the current defect type.

FIG. 11 depicts a user interface 1160 for providing a communicationportal with a consultant, according to some embodiments disclosedherein. More specifically, in response to selection of the chat with anconsultant options 976, 1076 in FIGS. 9 and 10, respectively, the userinterface 1160 may be provided. From the user interface 1160, the usermay conduct a video conference to discuss one or more aspects of theconsultation.

It should be understood that while in some embodiments, the userinterface 1160 is provided in response to selection of the chat with aconsultant options 976, 1076, this is merely an example. Morespecifically, if the consultation is a personal consultation, selectionof these options may call the consultant to physically walk over to theuser to provide the assistance.

FIG. 12 depicts a user interface 1260 for providing treatment and/orproduct recommendations to a user, according to some embodimentsdisclosed herein. As illustrated, once the image analysis is complete onthe user image, the service can review the information, provide productrecommendations, provide treatment recommendations, and/or provide otherservices. More specifically, the recommended products may be provided inthe user interface 1260, with purchase options 1262 a, 1262 b, and 1262c. Additionally provided in the user interface 1260 is a treatmentrecommendation for the subject.

Additionally, a simulated image showing an improvement and/or worseningto the defect areas may be provided. Simulating worsening may be usefulwhen the consultant is recommending a treatment using a product whichprevents skin degradation to show the user the potential affects ifhe/she fails to take precautionary measures. Simulating improvements maybe useful when the consultant is recommending a treatment using aproduct that eliminates and/or hides skin defects to show the analyzedperson the potential benefits of the products.

Similarly, in some embodiments, a text chat and/or other communicationsmay be provided for allowing the user to interact with the consultant.In some embodiments, a video recording option may be provided to allowthe user to save the consultation for later use.

Example Back-End Processing Windows

FIGS. 13A-13D depict a plurality of images that illustrates colorcorrection, according to some embodiments disclosed herein. Referringinitially to window 1342 in FIG. 13A, image processing may begin withreceiving an uploaded image and converting the image from RGB format toLAB format. In window 1344 and 1346, the B channel of the LAB convertedimage may then be filtered. By filtering the B channel between athreshold (e.g., 65-110), the blue coloring of the calibration device110 may be shown as white (binary 1) and the non-blue portions of theimage may be shown as black (binary 0). From this data, in FIG. 1348,the calibration device 110 in the RGB image may be located.Additionally, because the calibration device 110 may include thealignment indicator 252, two separate portions of the calibration device110 may be identified. Because of this, a measurement of the respectivelengths of the portions may be performed. The portion of the greatestlength is the portion of the calibration device of interest.

In the window 1350, the resolution indicator 254 may then be identified.The resolution indicator 254 may be identified by finding pixels in theLAB converted image that potentially could be the resolution indicator.From this another filtering may be performed to filter out theresolution indicator 254. Additionally in window 1350, a central pointof the resolution indicator 254 may be identified by determining thecoordinates of the pixels in the resolution indicator and determiningthe mean coordinate point. Additionally, to determine which arm of theresolution vector is the substantially vertical arm and which is thesubstantially horizontal arm, in window 1352 Eigen-Vectors may bedetermined from the central point to each pixel in the resolutionindicator 254. As the horizontal arm is longer, the direction with thelargest Eigen-Vector may be identified as the horizontal arm. In window1354 a boundary for separation of the resolution device 254 may bedetermined. In window 1356, the resolution indicator 254 may behighlighted in the RGB image.

Referring now to window 1358 in FIG. 13B, the RGB image may be retrievedand the previously determined horizontal arm may be separated from thevertical arm of the resolution indicator 254. In window 1362 a boundingbox may then be constructed around the horizontal arm of the resolutionindicator 254. The distance (in pixels) of the diagonal of the boundingbox may then be divided by the known actual length resolution indicator254 to determine the resolution of the image, as illustrated in window1366.

Once the resolution indicator has been determined, as discussed above,the calibration device 110 may be identified uniquely. Morespecifically, as there may be other objects in the uploaded image thatare the same (or similar) color as the calibration device 110, windows1368-1374 illustrate actions that may be used to remove those extraneousobjects. Referring now to window 1368 in FIG. 13C, a line that isperpendicular to the vertical arm of the resolution indicator 254 may bedetermined. From this line, a box may be created that spans apredetermined length in opposing directions from the perpendicular line.As an example, if the known width of the calibration device 110 is 3inches, the box may be created 1.5 inches on either side of theperpendicular line. Similarly, the box may span a length that beginsfrom the vertical line and extends ½ the known length of the calibrationdevice in opposing directions. From this box, the calibration device maybe identified as being within the boundaries of the box, as shown inwindow 1374. In window 1374, the RGB image may then be applied touniquely identify the calibration device 110.

As noted above, the calibration device 110 may include a plurality ofcolor chips. The plurality of color chips may be mapped from the imageto determine which values to retrieve from the database for comparison.Based on the observed values and the expected values, a color correctionmay be determined. Referring now to window 1376 in FIG. 13D, a templateof a color chip may be created. In window 1378, an angle of thecalibration device 254 in the uploaded image may be determined from thepreviously determined vertical arm. In window 1380, the template may berotated the determined angle. In window 1382, the template may beapplied to the uploaded image to determine possible color chips in theuploaded image. In window 1384, concentric circles may be created todetermine approximate locations of the actual color chips from theresolution indicator 254. In window 1386, a determination of the colorchips may be identified. In window 1388 the color chips may be numbered.In window 1390, based on the numbered color chips and the known color ofeach of the color chips, the image may be color corrected.

FIG. 14 depicts a plurality of images 1480 that illustrates finalresults from color correction. As illustrated, each of the images may bealtered according to the process described herein to provide an adjustedimage with standard image characteristics. More specifically, the images1482 were captured by a plurality of different image capture devices 104a, 104 b. As an example, image 1482 a was captured with the CanonPowershot A510; the image 1482 b was captured with the Canon PowershotS2 IS; the image 1482 c was captured with the Canon Powershot S3 IS; theimage 1482 d was captured with the Canon Powershot S70; the image 1482 ewas captured with the Canon Powershot SD550; the image 1482 f wascaptured with the KODAK EASYSHARE C743; the image 1482 g was capturedwith the KODAK CX6445; the image 1482 h was captured with the KODAKDX7590; and the image 1482 i was captured with the KODAK V705. As theimages 1482 were taken with different cameras, each of the images 1482may have different color characteristics. However, through the colorcorrection process described above, the images 1484 are consistent.Additionally, once the color correction has been performed, otheralterations to the images may be made, as described below.

As an example, the uploaded digital image may be compared to a mannequinimage to provide feature detection and spatial image adjustment. Assuch, the mannequin image may be utilized to determine relative headposition, relative head tilt, relative head pitch, and/or relative headyaw. The uploaded image may then be altered to substantially match themannequin image according to at least one image characteristic, such assize, orientation, position, etc., as described below with regard toFIGS. 15A and 15B.

FIGS. 15A and 15B depict a plurality of images that illustrates featuredetection and spatial image adjustment, according to some embodimentsdisclosed herein. As illustrated, the color corrected image in window1580 may be accessed for detecting at least one feature of the image.From the color corrected image, a two-tone facial mask image in window1582 may be created to determine various features on the targetsubstrate. The features may include a corner of the nostril, a corner ofthe eye, a corner of the mouth, and/or other features on the subject. Inwindow 1584, a first feature (e.g., the corner of the nostril) may bedetermined. This feature may be compared with a mannequin image (notillustrated) that has a corresponding first feature with a knownlocation.

Based on the known location of the first feature on the mannequin image,a search for a second feature on the uploaded image may be determined(e.g., a corner of the eye). The second feature may be located within aconstrained search area, which may be within a predetermined distancefrom the first feature, as determined from the mannequin image. If thesecond feature does not match the second feature on the mannequin image,the uploaded image (and/or the corresponding coordinate system) may bealtered (e.g., rotated, cropped, etc.) to allow the features to align.Similarly, in the window 1588, a third feature (e.g., a corner of themouth) may be determined as also being a predetermined distance from thefirst feature and/or the second feature, based on a corresponding thirdfeature of the mannequin image. In the window 1590, the image may becropped and/or rotated to match the characteristics of the mannequinimage. The altered image may then be provided in window 1592.

It will be understood that by comparing the uploaded image to amannequin image, the landmarks and features of the uploaded image may benormalized. More specifically, because many uploaded images are comparedwith a standard mannequin image, the alterations to the uploaded imageswill be consistent, thus providing a standard result by which to performspot and wrinkle analysis.

FIG. 15B depicts a plurality of color corrected images 1594, asdescribed above. Also included is a plurality of rotated images 1596,which have been compared with a mannequin image and rotated, based onthe determined features. A plurality of cropped images 1598 is alsoincluded, which provides the resulting image for spot and/or wrinkleanalysis, as described herein.

Example Front End Processes

FIG. 16 is a flowchart for providing one type of consultation. Asillustrated in block 1630, user data may be received from a kiosk 102,user computing device 118, and/or consultation center 112. At some pointduring (or after) the consultation, the user may wish to view and/orcontinue the consultation at another location. As such, data from theconsultation may be sent to the native remote computing device 114and/or foreign remote computing device 116. In block 1632, a request maybe received from a user computing device 118 (or kiosk 102) to continuethe consultation. In block 1634, the location of a first image may beretrieved, where the first image includes a calibration device 110.

In some embodiments, the first image may be stored on a foreign remotecomputing device 116, such as on a social network. The native remotecomputing device 114 may retrieve the image and determine that the imageincludes the calibration device 110. In block 1636, a determination maybe made regarding whether the color of the image has been adjusted,utilizing the calibration device 110. If not, at block 1638 the color ofthe first image may be determined and/or adjusted by utilizing thecalibration device 110. If the color has already been adjusted, in block1640 an analysis may be provided, a treatment may be recommended, aproduct may be recommended, and/or an option to purchase a product maybe provided.

In block 1642 a determination may be made regarding whether there is asecond image to utilize for the consultation. More specifically,oftentimes, the first image may cause the consultation to recommend aproduct and/or treatment. The user may then wish to return for anadditional consultation after utilizing the recommended product and/ortreatment. As such, the user may capture a second image so that theservice can analyze the improvement that the treatment and/or productcaused. Similarly, in some embodiments the user may desire to compare(or share with friends) images taken before, during, or after treatmentwith a cosmetic product in order to make self comparison of potentialchanges during the treatment period. If there is no second image, theprocess may end. If however, there is a second image, at block 1644 thesecond image may be retrieved, where the second image includes acalibration device 110.

In block 1646, a determination may be made regarding whether the colorof the second image has been adjusted. If not, in block 1648, the colorof the second image may be determined and/or adjusted utilizing thecalibration device 110. In block 1650, the first image may be comparedwith the second image. From this comparison, an additional analysis maybe performed, such as a treatment analysis. Additionally, a furtherproduct recommendation and/or treatment recommendation may be provided.An option to buy the recommended product and/or additional products mayalso be provided. As an example, the recommended products may includecolor cosmetic products, skin care products, hair care products, medicalproducts, dental products, grooming products, beauty and groomingdevices, and/or other products.

It should be understood that by saving the imagery and/or other data ona social network or other foreign remote computing device, options maybe provided for soliciting comments, feedback, ratings, and/or otherinformation from the social network community. Similarly, promotions,contents, and/or other events may be provided, which utilizes this data.

FIG. 17 depicts another flowchart for providing a virtual consultation,according to embodiments disclosed herein. As illustrated in block 1730,a kiosk 102 and/or user computing device may receive a first image of auser with a calibration device 110. In block 1732, the kiosk 102 and/oruser computing device 118 may utilize a consultation center 112 toprovide analysis and/or product recommendations. In block 1734, thekiosk 102 and/or user computing device 118 may receive the user logout.In block 1736, the kiosk 102 and/or user computing device 118 mayforward a first image to the foreign remote computing device 116. Inblock 1738, the user computing device 118 may utilize analysis logic 144b at the native remote computing device 114 to facilitate aconsultation. As discussed above, from the user computing device 118,the consultation may be a personal consultation, a semi-personalconsultation, and/or a virtual consultation. As such, the user computingdevice 118 may provide one or more interfaces for facilitating theconsultation.

In block 1740, the native remote computing device 114 may access thefirst image from the foreign remote computing device 116 for analysis.In block 1742, the native remote computing device 114 may utilize thecalibration device 110 to determine and/or adjust the color of the firstimage. In block 1744, the native remote computing device 114 may sendthe first image to the user computing device 118 and may produce aninterface for providing the consultation. In block 1746, the usercomputing device 118 receives a second image that includes a calibrationdevice 110. In block 1748, the user computing device 118 utilizes thenative remote computing device 114 and the calibration device 110 todetermine and/or adjust the color of the second image. In block 1750,the user computing device 118 may utilize the native remote computingdevice 114 to compare the second image with the first image and providean analysis, a product recommendation, a treatment recommendation,and/or an option to purchase a product. In some embodiments, the firstand second images may merely be displayed on the user computing device118 for visual inspection by the user.

FIG. 18 depicts a flowchart for providing a recommendation to a user,according to embodiments disclosed herein. As illustrated in block 1830,a request for a consultation may be received. In block 1832, user datamay be received, where the user data includes an image of a targetsubstrate. The image may include a calibration device 110. In block1834, the image may be altered according to predetermined standards,based on the calibration device 110. In block 1836, an analysis may beperformed of the user data. In block 1838, a recommendation may beprovided, based on the analysis.

FIG. 19 depicts another flowchart for providing a recommendation to auser, according to embodiments disclosed herein. As illustrated in block1930, data related to a previously established consultation may bereceived. The data may include a first image of a target substrate. Thedata may also include information related to a first recommendation ofthe previously established consultation. In block 1932, the first imagemay be sent to a foreign remote computing device 116 for storage. Inblock 1934, a request to resume the previously established consultationmay be received. In block 1936, the first image from the foreign remotecomputing device 116 may be accessed. In block 1938, a second image ofthe target substrate may be accessed, the second image being capturedafter the first image. In block 1940, the second image may be altered,based on predetermined standards to provide substantially similar imagecharacteristics as the first image. In block 1942, the first image maybe compared with the second image to determine progress of thepreviously established consultation. In block 1944, a secondrecommendation may be provided to a user.

FIG. 20 depicts a flowchart for capturing an image of a user and acalibration device 110, according to some embodiments disclosed herein.As illustrated in block 2030, an image capture device may be activated.In block 2032, a calibration device 110 may be positioned on the user,where the boundary marker is centered on the user's head. In block 2034,the user may be positioned within the field of view of the image capturedevice 104 a, 104 b and rotated at a predetermined angle to providevisual access to the target substrate. In block 2036, an image may becaptured with user's eyes open, in a relaxed position, and optionallywith or without wearing makeup. The image may be transmitted via thenetwork 100 to the consultation center 112 and/or the native remotecomputing device 114. In block 2038, image properties may be adjusted,using calibration device as a guide. In block 2040, the image may beanalyzed and/or altered to facilitate the consultation. For example, theimage may be color corrected and/or facial features (e.g., fine lines,wrinkles, age spots, crow's feet) may be identified therein by one ormore indicators 964, 1064 (see, e.g., FIGS. 9 and 10), such as a box,circle, line, arrow, or other geometric shape or symbol. The alteredimage of the user containing the indicators may be transmitted from theconsultation center 112 and/or the native remote computing device 114 tothe kiosk 102 or a user computing device 118 via the network 100 anddisplayed on a display device thereof.

Example Back-End Processes

FIG. 21 depicts a flowchart for altering an image for analysis,according to some embodiments disclosed herein. Similar to thediscussion from FIG. 14 above, in block 2130, an image of the subjectmay be received. As discussed above, the image may be received from animage capture device 104 a, 104 b, from a foreign remote computingdevice 116 (such as a social network), from the user computing device118, and/or from another source. In block 2132, a color correction ofthe image may be performed. In block 2134 a spatial image adjustment maybe performed.

More specifically, as described above, the uploaded digital image may becompared to a mannequin image. The mannequin image may have apredetermined relative head position, relative head tilt, and/orrelative head yaw. The uploaded digital image may then be compared withthe mannequin image. If the uploaded digital image does not match themannequin image, the uploaded digital image may be altered tosubstantially match the mannequin image according to at least one imagecharacteristic, such as size, orientation, position, etc. In someembodiments, altering the uploaded digital image includes matching apredetermined point on the portion of the face of the subject with acorresponding point on the portion of the face of the mannequin. Inblock 2136, the image may be compared to a mask to further adjust thecharacteristics of the uploaded image. In block 2138, feature analysismay be performed.

FIG. 22 depicts a flowchart for performing color correction, accordingto some embodiments disclosed herein. As illustrated, from FIG. 21, theblock 2132 may include a plurality of actions. More specifically, inblock 2230 a calibration device detection may be performed. In block2232, the resolution indicator 254 may be detected. In block 2234 theresolution indicator may be separated into a horizontal arm and avertical arm. In block 2236, image resolution may be estimated from theresolution indicator. More specifically, the horizontal arm of theresolution indicator 254 may have a predetermined length. Based on thenumber of pixels that the resolution indicator 254 spans in the imagethe resolution of the image may be determined.

Additionally, determinations may be made regarding whether the uploadedimage meets resolution thresholds. More specifically, in someembodiments the image should be at least about 220 pixels per inch (andin some embodiments between about 150 to about 500 pixels per inch),which would be a predetermined first resolution threshold. Thisthreshold has been determined as being adequate for performing wrinkleand/or spot analysis. If the uploaded image meets that requirement, theimage may be processed as described herein. If not, a determination maybe made regarding whether the uploaded image is of a predeterminedsecond resolution threshold that is available for up-conversion. If not,the image may be rejected. If so, the image may be up-converted to meetthe 220 per inch threshold.

Returning to FIG. 22, in block 2238 the calibration device 110 may beisolated uniquely. As discussed above, while detecting the calibrationdevice 110 in block 2230 may locate the calibration device, otherobjects in the image may have the same color as the calibration device110 and may be mistakenly identified as well. As such, block 2238utilizes the determined resolution indicator 254 to remove anyextraneous objects that may be identified as the calibration device 110.In block 2240, a phase 1 color correction may be performed. In block2242, a phase 2 color correction may be performed. In block 2244, aphase 3 color correction may be performed.

FIG. 23 depicts a flowchart for detecting a calibration device,according to some embodiments disclosed herein. As illustrated in theblock 2230 of FIG. 22, calibration device 110 was detected. The blocksof FIG. 23 further elaborate on the device detection of FIG. 22. Morespecifically, in block 2330, the uploaded image may be converted fromRGB format to LAB format. In block 2332, pixels in the B channel thatare within a minimum and maximum threshold may be found. In block 2334,false positive pixels may be filtered by computing the major axis lengthand eccentricity of an equivalent ellipse.

FIG. 24 depicts a flowchart for detecting a resolution indicator,according to some embodiments disclosed herein. More specifically, inblock 2232 from FIG. 22, a plurality of actions may be performed. Morespecifically, in block 2430, the LAB image may be retrieved (e.g. frommemory). In block 2432, the calibration device (RGB) image from block2334 may be retrieved. In block 2434, each of these images may beanalyzed to find pixels that satisfy the threshold constraints of bothof the RGB image and the LAB image, as well as being located within apredetermined area for the calibration device 110. As discussed above,the LAB image determines thresholds for potential calibration devices.The calibration device image from 2334 may be used to filter outportions that are not in the appropriate area for the calibration device110.

FIG. 25 depicts a flowchart for separating a resolution indicator,according to some embodiments disclosed herein. More specifically, fromFIG. 22, the block 2234 may be expanded into a plurality of actions. Asillustrated in block 2530, a central point of the resolution indicatormay be found. As discussed above, the central point may be found bydetermining the coordinates of the pixels in the resolution indicator254. These coordinates may then be averaged, which yields the centralpoint.

In block 2532, an Eigen-analysis of points in the resolution indicatorspace may be performed. More specifically, to determine which arm of theresolution indicator 254 is the horizontal arm, the Eigen-Vectors may bedetermined from the central point to each pixel in the resolutionindicator 254. The Eigen-Vectors are then utilized to determine adirection of maximum radiation. From this determination, the horizontalarm (which in this example is longer) may be determined. In block 2534,an orientation of the points with respect to the largest Eigen-Vectormay be found. In block 2536, points that have an orientation that liesbetween a predetermined boundary of the calibration device 110 may befound.

FIG. 26 depicts a flowchart for performing image resolution estimation,according to some embodiments disclosed herein. More specifically, block2236 from FIG. 22 may include a plurality of actions. As illustrated inblock 2630, an image that includes the resolution indicator 254 may bereceived (e.g., the image from block 2536). In block 2632, a boundingbox of the resolution indicator 254 may be determined. In block 2634, alength of the diagonal of the bounding box may be computed by countingpixels of the diagonal. In block 2636, an estimated image resolution maybe determined by dividing the length of the diagonal by a predeterminedlength value, (e.g., 1 inch).

FIG. 27 depicts a flowchart for isolating the calibration deviceuniquely, according to some embodiments disclosed herein. Morespecifically, from block 2238 in FIG. 22, a plurality of actions may beperformed. As illustrated in block 2730, the resolution indicator 254may be identified. Similarly, in block 2732, the estimated imageresolution may be identified (from FIG. 26). In block 2734, a first linethat has a common orientation with the vertical arm of the resolutionindicator may be computed. In block 2736, a second line that isperpendicular to the vertical arm may be computed. In block 2738, thefirst line may be moved a second predetermined distance in the +/−direction along the perpendicular line.

More specifically, if the calibration device 110 has a known length of 7inches, the predetermined distance may be 3.5 inches in each direction.In block 2740, the second line may be moved a second predetermineddistance in the +/− direction at two locations that are parallel withthe vertical arm. So, if the calibration device 110 has a known width of3 inches, the predetermined distance would be 1.5 inches in eitherdirection. These predetermined distances should provide a bounding boxthat approximately outlines the calibration device 110. In block 2742,the bounding box may be created using estimated locations. In block 2744the calibration device image may be received. In block 2746, anintersection area between the bounding box and the calibration devicemay be found.

FIG. 28 depicts a flowchart for performing phase 1 image colorcorrection, according to some embodiments disclosed herein. Morespecifically, from block 2240 in FIG. 22, the phase 1 image colorcorrection may include a plurality of actions. As illustrated in block2830, a template for a calibration device 110 may be created. Thetemplate may identify the location and color of various portions of thecalibration device 110. In block 2832, the vertical arm of theresolution indicator 254 may be received. In block 2834, the verticalarm may be utilized to rotate the template (which is image specificrotation). In block 2836, the calibration device image from block 2330may be received. In block 283, the calibration device location may beutilized to search for candidate color chips by matching the calibrationdevice image with the template.

FIG. 29 depicts a flowchart for performing phase 2 image colorcorrection, according to some embodiments disclosed herein. Morespecifically, from block 2242 in FIG. 22, a plurality of actions may beperformed in phase 2 color analysis. As illustrated in block 2930,potential colored region candidates may be received. In block 2932, thecolor chips that are located in concentric circles from the center ofthe resolution indicator 254 may be found. In block 2934, the colorchips may be sorted in each quadrant in a predetermined direction. Inblock 2936, the inter-point distances between the color chips may becomputed. More specifically, in the first concentric circle, the colorchips closest to the resolution indicator 254 may be identified. In thesecond concentric circle, the color chips outside the closest colorchips may be determined. Additionally, inter-point distances may bedetermined between each of the first tier color chips and each of thesecond tier color chips. In block 2938, the missing chip locations maybe predicted utilizing known and determined special information. Inblock 2940, the color chips may be reordered for calibration devicestate estimation.

FIG. 30 depicts a flowchart for performing phase 3 image colorcorrection, according to some embodiments disclosed herein. Morespecifically, from block 2244 in FIG. 22, the phase 3 image colorcorrection may include a plurality of different actions. As illustratedin block 3030, the ordered colored region locations may be received. Inblock 3032, the estimated color region values may be extracted from theimage from block 2330. In block 3034, the expected color region valuesmay be received. In block 3036, the similarity between the reorderedcolor chips with the expected color chips may be computed. In block3038, the estimated colored region values may be reordered for thecalibration device 110. In block 3040, the reordered color chips may bedetermined, according to the best calibration device state.

FIG. 31 depicts a flowchart for performing phase 4 image colorcorrection, according to some embodiments disclosed herein. Morespecifically, within block 2246 in FIG. 22, a plurality of actions maybe performed. In block 3130, the expected color chip values may bereceived from FIG. 30. In block 3132, the expected color chip values maybe converted from RGB format to LAB format. In block 3134, the L channelmay be used to compute the intensity scale factor of the expected colorchips.

Similarly, in block 3136, the estimated color chip values may bereceived. In block 3138, the estimated color chip values may beconverted from RGB to LAB format. In block 3140, the A and B channelsmay be used to compute a color transformation of the estimated colorchips. In block 3142, a single LAB to LAB color transformation may beperformed to create a matrix of color transformation values. In block3144, the transformation values may be applied on the uploaded image tocreate a color corrected image. Once this is complete, the process mayend and/or continue in block 2134 from FIG. 21.

It should be understood that while the blocks 3130-3134 and 3136-3140are depicted as being performed in parallel, this is merely an example.More specifically, these and other blocks described herein may beperformed in a different order than depicted herein without departingfrom the intended scope of this disclosure.

In summary, persons of skill in the art will readily appreciate thatsystems, methods, and devices for providing products and consultations.The foregoing description has been presented for the purposes ofillustration and description. It is not intended to be exhaustive or tolimit the invention to the example embodiments disclosed. Manymodifications and variations are possible in light of the aboveteachings. It is intended that the scope of the invention not be limitedby this detailed description of example embodiments, but rather by theclaims appended hereto.

All documents cited in the Detailed Description of the Invention are, inrelevant part, incorporated herein by reference; the citation of anydocument is not to be construed as an admission that it is prior artwith respect to the present invention.

While particular embodiments of the present invention have beenillustrated and described, it would be obvious to those skilled in theart that various other changes and modifications can be made withoutdeparting from the spirit and scope of the invention. It is thereforeintended to cover in the appended claims all such changes andmodifications that are within the scope of this invention.

1. A method for image analysis, comprising: accessing a digital image ofa subject, the digital image comprising a portion of a target substrateof the subject, the digital image comprising a portion of a calibrationdevice; detecting a position of the calibration device by searching thedigital image for a predetermined background color; detecting aplurality of color chips on the calibration device; mapping a positionof the plurality of color chips on the digital image, each of theplurality of color chips comprising a predetermined chip color;comparing at least a portion of the plurality of color chips with apredetermined color standard; and altering the digital image, based onthe comparison, to provide consistent image parameters with other imagesof the subject.
 2. The method of claim 1, comprising transmitting thedigital image to a remote computing device.
 3. The method of claim 1,comprising transmitting the altered digital image to a user computingdevice.
 4. The method of claim 1, comprising storing at least one of thedigital image, the altered digital image, data associated with alteringthe digital image and data associated with the subject.
 5. The method ofclaim 1, comprising: receiving data related to the subject; retrievingstandard image characteristics that correspond with the data related tothe subject; and utilizing the standard image characteristics to alterthe digital image.
 6. The method of claim 5, wherein the data related tothe subject comprises at least one of the following: an age of thesubject, a zip code of the subject, an ethnicity of the subject, agender of the subject, and a skin type of the subject.
 7. The method ofclaim 1, wherein the altered digital image comprises one or moreidentifiers identifying a condition.
 8. The method of claim 7, whereinthe condition is located in at least one of the following areas: an areaadjacent an eye and an area of a cheek of the subject.
 9. The method ofclaim 1, wherein the altered digital image is altered by at least one ofa color correction and a spatial image adjustment.
 10. The method ofclaim 9, wherein the spatial image adjustment includes at least one ofthe following: scaling, centering, relative pitch, and relative headyaw.
 11. The method of claim 1, comprising providing for displayinganalysis data associated with the altered digital image.
 12. The methodof claim 1, wherein the digital image of the subject includes theportion of the target substrate captured from a first angle, the methodcomprising: comparing the digital image with a mannequin image, themannequin image comprising at least a portion of a target substrate of amannequin from an angle that is substantially similar to the firstangle; and altering the digital image to substantially match themannequin image according to at least one image characteristic.
 13. Themethod of claim 12, wherein the at least one image characteristiccomprises at least one of the following: image resolution, faceposition, face size, face tilt, relative face pitch, and relative headyaw.
 14. The method of claim 12, wherein altering the digital imagecomprises matching a feature on the portion of the target substrate ofthe subject with a corresponding feature on the portion of the targetsubstrate of the mannequin.
 15. The method of claim 14, wherein thefeature on the target substrate includes at least one of the following:a corner of an eye, a corner of a mouth, and a corner of a nostril. 16.The method of claim 1, comprising: detecting a resolution indicatorwithin the position of the calibration device; determining an imageresolution of the digital image from the resolution indicator; anddetermining a calibration device angle of the calibration device on thesubject, the calibration device angle being determined from theresolution indicator.
 17. The method of claim 16, comprising:determining whether the image resolution meets a predetermined firstresolution threshold; if the digital image does not meet thepredetermined first resolution threshold, determining whether thedigital image meets a predetermined second resolution threshold; if thedigital image meets the predetermined second resolution threshold,up-converting the digital image to meet the predetermined firstresolution threshold; and if the digital image does not meet thepredetermined second resolution threshold, rejecting the digital image.18. The method of claim 17, comprising: determining whether the digitalimage exceeds the predetermined first resolution threshold; and if thedigital image exceeds the predetermined first resolution threshold,scaling-down the digital image to the predetermined first resolutionthreshold.
 19. A system for image analysis, comprising: a memorycomponent that stores logic that, when executed by the system, causesthe system to perform at least the following: access a digital image ofa subject, the digital image comprising a portion of a target substrateof the subject, the digital image comprising a portion of a calibrationdevice; detect a position of the calibration device by searching thedigital image for a predetermined background color; detect a pluralityof color chips on the calibration device; map a position of theplurality of color chips on the digital image, each of the plurality ofcolor chips comprising a predetermined chip color; compare at least aportion of the plurality of color chips with a predetermined colorstandard; and alter the digital image, based on the comparison, toprovide consistent image parameters with other images of the subject.20. The system of claim 19, wherein the logic causes the system todetect at least one feature on the portion of the target substrate,wherein detecting at least one feature on the portion of the targetsubstrate comprises: creating a two-tone facial mask image; andidentifying pixels in the image that correspond to the portion of thetarget substrate.
 21. The system of claim 19, wherein the logic causesthe system to analyze the image to identify at least one condition inthe portion of the target substrate.
 22. The system of claim 19, whereinthe logic causes the system to compare the digital image to a mannequinimage to determine at least one of the following: relative headposition, relative head tilt, relative head pitch, and relative headyaw.
 23. The system of claim 19, wherein the logic causes the system toperform at least the following: detect a resolution indicator within theposition of the calibration device; determine an image resolution of thedigital image from the resolution indicator; and determine a calibrationdevice angle of the calibration device on the subject, the calibrationdevice angle being determined from the resolution indicator.
 24. Thesystem of claim 23, wherein the logic causes the system to perform atleast the following: determine whether the image resolution meets apredetermined first resolution threshold; if the digital image does notmeet the predetermined first resolution threshold, determine whether thedigital image meets a predetermined second resolution threshold; if thedigital image meets the predetermined second resolution threshold,up-convert the digital image to meet the predetermined first resolutionthreshold; and if the digital image does not meet the predeterminedsecond resolution threshold, reject the digital image.
 25. The system ofclaim 24, wherein the logic causes the system to perform at least thefollowing: determine whether the digital image exceeds the predeterminedfirst resolution threshold; and if the digital image exceeds thepredetermined first resolution threshold, scale-down the digital imageto the predetermined first resolution threshold.